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Catalyst schools were 28 elementary and secondary 
schools selected to participate in a pilot project begun 
in July 2014, which explored how best to support 
teacher professional learning through decentralization 
of decision making and implementation of the Learning 
School approach. The pilot project was the first phase in 
a statewide initiative that would be scaled up to all public 
schools in the state in the 2016-2017 school year. 

Learning Forward, a national professional learning 
association, developed the Learning School approach 
as a model for implementation of the Standards for 
Professional Learning. The standards make up a 
framework for continuous school improvement based 
on ongoing educator learning to support positive 
student outcomes. The West Virginia Board of Education 
(WVBE) adopted the standards in 2012; the Learning 
School Initiative was the Board’s call for systematic 
implementation of the standards. 

This report focuses on levels of implementation of the 
Learning School approach achieved by catalyst schools 
across the state. It is the fourth in a series of reports 
based on a research study, “Developing Effective 
Professional Learning Communities in Catalyst Schools,” 
conducted between February 2015 and June 2016. 


Variation in Levels of Implementation 

Focus group interviews with principals in late 2015 and 
early 2016 revealed that catalyst schools were having 
a wide range of experiences in the Learning School 
Initiative. Most catalyst schools were selected because 
they already had many of the Learning School elements 
in place, especially professional learning communities 
(PLCs) scheduled during the regular school day and 
an ongoing practice of using student data to guide 
instruction. 

The principal interviews provided evidence that some 
schools had embraced the Learning School approach 
and were using it to further improve teaching and 
learning. Administrators from a few other catalyst 
schools did not find the orientation session held in July 
2015 very useful or beneficial, and appeared not to 
have engaged with the Learning School approach to 
any significant degree. Other principals interviewed 
during this time period were just beginning to turn their 
attention to implementing the Learning School approach 
after spending the first half of the 2015-2016 school 
year preparing for visits by the West Virginia Office 
of Education Performance Audits (OEPA), the school 
monitoring and accreditation agency in the state. 


Likewise, some regional education service agencies (RESAs) — the agencies assigned responsibility for supporting 
implementation of the Learning School approach — appeared to be fully engaged in supporting their catalyst schools, 
while others remained focused on OEPA audits and other priorities. 

To develop a clearer picture of this variation among schools and RESAs, this study focused on the interpretation of five 
main data sources as measures of implementation: 

1. Time allotted in schedules for PLCs. 

2. Types of activities included in PLC agendas. 

3. Standard Assessment Inventory 2 (SAI2) scores. 

4. Focus group interviews with principals. 

5. RESA and the West Virginia Department of Education 's (WVDE) assessments of each school’s stage of 
implementation. 

This analysis assigned ratings on a 5-point scale for four of these measures (Items 1-3, 5), with 1 representing a very low 
level of implementation and 5 representing a very high level. In the following sections, I describe how those ratings were 
derived and display aggregate findings at the state and RESA levels. The Item 4 measure was used to supplement the 
SAI2 measure (see Item 3) and was not included in the scoring. 

Note: RESAs have been given randomly assigned pseudonyms in all of the analyses in this report to protect the 
confidentiality of the catalyst schools. For additional information about methods used in this study see, Catalyst Schools 
Research Study Technical Report, found on the WVDE Office of Research, Accountability, and Data Governance website 
at http://wvde.state.wv.us/research/reports2016.html. 


Time allotted in schedules for 
PLCs 


Time is the starting point; without 
sufficient time in schedules, it 
is difficult to have success as a 
Learning School. In response to a 
request, twenty-five catalyst schools 
provided information about the 
schedules for their PLCs. From that 
information I calculated the number 
of minutes typical PLCs met per 
month at each catalyst school. I then 
derived a 5-point scale, with the top 
of this scale (5 points) approaching 
the amount of time recommended in 
the literature (45 minutes, 4 days a 
week) 1 and the bottom of the scale 
equaling less than one 45-minute 
sessions every two weeks (see 
Figure 1 for additional details about 
the scale). 

As Figure 1 shows overall, PLCs in 
catalyst schools were meeting for 
45 minutes, less often than 1 time 
a week — a rate far lower than that 
called for in the research literature 
(see Discussion section). 2 


5.0 


.5? 



Catalyst schools mean score by RESA and overall 


Figure 1. Mean Rating for Amount of Time Scheduled for PLCs Overall and by RESA 

Data source: PLC schedules supplied by catalyst school principals. Scale: 5 = 720+ (min. 4 days/ 
week @ 45 minutes/day or more); 4 = 360 - 719 (min. = 2 days/week @ 45 minutes/day); 3 = 180 - 
359 (min. = 1 day/week @ 45 minutes/day); 2 = 90 - 179 (min. = 2 day/month @ 45 minutes/day); 1 
= 0-89 (less than 2 day/month @ 45 minutes/day). 
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This finding aligns with the oft-cited concern in focus group settings that there 
simply was not enough time for PLCs to meet. The average overall score of 
2.2 obscures the considerable range of experiences, but this figure does point 
to the need for more support in developing schedules and calendars that will 
reserve more time for professional learning. The average scores, reported in 
Figure 1 by RESA, give a picture of the range among catalyst schools. 


Types of activities included in PLC agendas 

Even with time available, the way the time is used is crucial to the quality of 
professional learning. Eighteen schools responded to a request for copies 
of their PLC agendas for the month of February 201 6. 3 The agendas were 
anonymized and agenda items were numbered and categorized separately 
by two individuals, using the categories shown in the sidebar (next page). 
When categories assigned to a particular agenda topic differed between 
the two raters, a discussion ensued and consensus was reached about the 
appropriate category. 


Overall 
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Figure 2. Percent of Catalyst School PLC Agenda Items by Category of Activity and RESA 
Data source: PLC agendas submitted for the month of February 2016 by 18 catalyst schools. 

For each school, percentages of agenda topics were calculated for each of the 
major categories. Those percentages were then averaged across the state and 
individual RESAs. 

Figure 2 shows that the large majority of agenda items fell into the planning 
for learning category. Not shown here, but present in the more detailed data 
was evidence that the vast majority of that planning was focused on student 
learning, with a small percentage focused on planning for teacher learning. 

In the 5-point scoring scheme (needed for the final composite score), agenda 
items were weighted; those focused on professional learning were weighted 
1 .5 points because they are the most desirable activities to be taking place 
during PLCs. Based on the assumption that there is some degree of learning 
and sharing that takes place when collaborative teams get together to review 
student data and make plans for student learning, 1 point was given for each 
planning item. A negative one quarter point (-.25) point was assigned for each 
percentage point of administrative tasks more appropriately handled in faculty 
meetings or other modes of communication. 


Categories of Activities 
Included in Catalyst School 
PLC Agendas 

Administrative information 
sharing 

• Logistics/schedules— Field 
trips, school events, bus 
schedules, etc. 

• Procedures — Management, 
GSA testing, WVEIS 
reporting, fire drills, and other 
related topics 

• News/info — Change from 
NCLB to ESSA, school board 
decisions, etc. 

• Other — Additional 
administrative topics 

Planning for learning 

• Student learning — Reviewing 
individual and group-level 
student academic and 
early warning data, making 
adjustments, monitoring 
progress, planning PBL 
projects, and other related 
topics 

• Professional learning — 
Planning for teacher 
learning needs based on 
student needs and based 
on educator evaluation, I PI , 
SAI2, OEPA data and other 
data 

• Other planning for learning — 
Additional planning topics 

Engaging in professional learning 

• Student behavior/school 
climate/parent involvement — 
PBIS, school climate and 
culture, parent involvement, 
attendance matters, using 
the early warning data, and 
other related topics 

• Content/pedagogy— Content 
area, instruction, formative 
assessment, how to use 
student data, using new 
curriculum materials, etc. 

• Professional learning 
systems — Becoming a 
Learning School, effective 
PLCs, IPI training, etc. 

• Other — PD on other topics 
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After the total weighted percent was calculated for each 
school (the highest possible score was 150), scores were 
assigned according to the scale shown in Figure 3. 
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Catalyst schools by RESA and overall 


Figure 3. Mean Rating for Quality of PLC Agenda Topics by RESA and 
Overall 

Data source: PLC agendas provided by catalyst schools for the 
month of February 2016. Scale: 5 = at least 90% of items focused 
on professional learning and planning with strong emphasis on 
professional learning; 4 = at least 90% of items focused on professional 
learning and planning with emphasis on planning; 3 = at least 70% of 
items focused on professional learning and planning with emphasis on 
planning and many items focused on administrative tasks; 2 = about 
60% or less of items focused on professional learning and planning 
combined; 1 = nearly all items focused on administrative tasks. 

As we have seen in other measures, RESAs varied 
considerably in terms of the quality of agenda topics 
included in their catalyst school PLC agendas. Overall, 
though, catalyst schools were at the middle of the 
5-point scale, indicating that their agendas tended to 
focus mostly on planning, with a secondary focus on 
professional learning, and as much as 30% of their 
agendas taken up with administrative items. 


Focus on Standards for Professional Learning 

Two sets of data were collected to capture the extent 
to which catalyst school teachers and principals were 
focused on all seven of the Standards for Professional 
Learning at the catalysts schools. Described first are the 
results of a survey of teachers and second, the results of 
focus groups conducted with principals. 

Alignment with Standards for Professional 
Learning 

The SAI2, a valid and reliable instrument developed by 
Learning Forward, provides a measure of the extent to 
which professional learning practices in catalyst schools 
aligned with the Standards for Professional Learning. 
Only 10 of the 28 catalyst schools took the SAI2 in 
spring 2016 with a sufficient level of participation to 
measure the extent to which their professional learning 
practices aligned with the Standards for Professional 
Learning. Each standard receives a score on a scale 


of 1 (low) to 5 (high). For the purposes of this study, an 
average composite score was calculated for each of the 
10 schools. Results aggregated at the RESA level and 
across the state are shown in Figure 4. With nearly two 
thirds of the schools not participating in the SAI2, three 
RESAs did not have results that could be shown here. It 
would be reasonable to speculate that schools choosing 
to participate were also otherwise strongly engaged in 
the Learning School implementation process; accordingly 
their scores may not be representative of catalyst schools 
overall or within their RESAs. The low level of catalyst 
school participation in the spring 2016 administration 
of the SAI2 may also be indicative of not having had 
the opportunity to experience the benefits of using the 
SAI2 results from the spring 2015 administration of the 
survey. At that time all five Cohort 1 schools took the 
survey and 20 of the 23 Cohort 2 schools also took the 
survey. The Cohort 2 schools were shown their results at 
the orientation sessions in July 2015. However, several 
catalyst school principals mentioned have difficulty 
accessing their results after the orientation, and only a 
small minority reported using their SAI2 results in their 
planning during focus group interviews, even when 
specifically asked about it. 
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Catalyst schools by RESA and overall 


Figure 4. Alignment With Standards: Mean Catalyst Schools SAI2 Score 
(All Seven Standards) by RESA and Overall 

Data source: Learning Forward’s SAI2 online survey results for 10 
catalyst schools. Scale: 1 = low level of alignment; 5 = high level of 
alignment. 


This result among catalyst schools may sound a 
cautionary note about the importance of working with 
the schools just beginning to engage with the Learning 
School approach as part of the statewide expansion of 
the program. Several hundred schools have requested 
participation in the SAI2 in advance of the 2016-2017 
school year; it will be important to make sure they are 
able to readily access the data and that they experience 
the usefulness of the data in their planning for 201 6-201 7. 
Lacking that kind of support, they may be reluctant to use 
the survey again during subsequent years, as were about 
two thirds of the catalyst schools. 
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Focus Group Interviews with Catalyst School Principals 
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Catalyst school principals 
participated in focus group 
interviews in late 2015/early 2016, 
during which they were asked a 
variety of questions, one of which 
focused specifically on the Standards 
for Professional Learning. They were 
given a list of the seven standards 
and asked to describe any activities 
in their schools that were intended 
to implement individual standards. 

Responses varied greatly across 
the RESAs, but overall results are 
shown in Figure 5. Perhaps not 
surprisingly for interviews with 
principals, leadership was the 
standard mentioned most frequently 
(19.1%), followed by learning 
designs (17.4%), and outcomes 
(17.4%). The leadership standard 
includes developing leadership capacity among teachers, advocating for professional learning, and creating systems 
and structures (e.g., schedules and calendars) to support teacher learning. Catalyst school principals’ focus on learning 
designs includes aligning professional learning with teacher experience/needs, using a variety of forms (e.g. book 
studies, active research), and using peer observation. The principals also talked frequently about such student outcome- 
related issues as focusing on curriculum and how students learn, building on previous teacher learning, and explicitly 
linking educator and student learning. These appear to be important strengths among catalyst schools, according to 
their principals. 


Overall 

Professional-learning-standard themes mentioned by catalyst schools 

Figure 5. Catalyst School Principals' Mentions of Individual Professional Learning Standards as 
Percentage of Their Total Mentions of Standards During Focus Groups 

Data Source: Eight focus group interviews conducted with catalyst school principals during late 
2015/early 2016. 


On the other hand, mentions of the standard, implementation, lagged far behind the other standards (4.7% of all 
mentions of standards). This standard includes activities such as applying research on change, sustaining support 
for long-term change, and providing constructive feedback. Tied for second lowest were two standards (13.5%): 
data and resources. The low percentage of mentions of the use of data may seem surprising; however the Standards 
for Professional Learning extend beyond using formal and informal student assessment data — which were often 
mentioned — to other sources of data such as educator data (e.g., educator evaluations, SAI2) and system data (e.g., 
OEPA, early warning, school climate, fiscal). A focus on resources would include such things as open discussions of 
expenditures for professional learning, allocating time for professional learning during the school day, and providing 
technology and other professional learning resources. 

These findings point to some areas where catalyst schools could benefit from additional capacity building and where 
RESAs and others may need to focus as the WVBE’s Learning School initiative moves forward. 
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Stage of Implementation Scores 

According to the implementation science literature, initiatives pass through a progression of stages as described below: 4 

Stage 1 = Exploration — identifying the need, acquiring information, assessing the fit between the need and 
intervention program, and preparing the organization by mobilizing information and support 

Stage 2 = Installation — making preparations, such as human resource strategies (e.g., scheduling time), policy 
development, reporting frameworks, outcome expectations, staff training, and needed technology/resources 

Stage 3 = Initial implementation — beginning to use and monitor new practices, developing and applying new 
skills, creating a supportive organizational culture, and overcoming inertia and resistance 

Stage 4 = Full implementation — proceeding with innovation as accepted practice and routine, and seeing 
evidence of expected impacts/improvements 

Stage 5 = Sustainability — maintaining the effectiveness of the innovation as staff come and go and other changes 
take place in the organization and its environment 
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Catalyst schools by RESA and overall 


Figure 6. Mean Stage of Implementation for 
Catalyst Schools Overall and by RESA 

Data source: Aggregate stages 
implementation, assessed by two individuals 
for each of 28 catalyst schools, one from 
WVDE and one from the associated RESA. 
Scale: 1 = exploration; 2 = installation; 3 = 
initial implementation; 4 = full implementation; 
5 = sustainability. 


One WVDE staff member and one RESA staff member independently assigned scores to each of the 28 catalyst 
schools for the stage each school had reached in its implementation of the Learning School approach. The individuals 
assigning the scores had firsthand familiarity with the schools to which they assigned ratings. As shown in Figure 6, 
catalyst schools have been assessed overall as approaching the initial implementation stage. However, this figure also 
shows great variability among the RESAs, with schools in two RESAs rated as still in the first stage of implementation 
(exploration) and schools in two other RESAs well into the initial implementation stage-in one case approaching full 
implementation of the Learning School approach in their catalyst schools. 
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How PLC Time is Used 


Composite Scores: Mean of all Previous Scale Scores 

Composite ratings for individual catalyst schools were calculated by taking the 
average of the scale scores described previously for PLC time in schedules, 
PLC agenda quality, SAI2 standards alignment, and stages of implementation. 
Schools for which at least three of four ratings were available were included in 
the aggregated information shown in Figure 7. This figure provides a portrayal 
of the level of implementation in catalyst schools across West Virginia. The two 
RESAs with the highest scores are spotlighted in the second and third reports 
in this series. 5 
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Catalyst schools by RESA and overall 


Figure 7. Composite Level of Implementation Ratings by RESA and Overall 

Data sources: Aggregate mean ratings for (a) amount of time scheduled for PLCs; (b) mean rating 
for quality of PLC agenda topics; (c) mean SAI2 Scores (all seven standards); and (d) mean stage 
of implementation for catalyst schools — calculated by RESA and overall. 

Scale: 1 = low level of implementation; 3 = moderate level of implementation; 5 = high level of 
implementation. 


Discussion 

Time for Professional Learning 

Overall, catalyst schools were scheduling far less embedded learning time for 
their PLCs-on average, 90 - 179 minutes a month-than 720 minutes called 
for in research. According to Learning Forward, “Dedicated job-embedded 
learning time elevates the importance of continuous, careerlong learning as a 
professional responsibility of all educators and aligns the focus of their learning 
to the identified needs of students they serve.” 6 Not only is such scheduled 
time better aligned with teachers’ and students’ learning needs, it can also 
be an efficient use of funding: “Including substantive time for professional 
learning, 15% or more, within the workday shifts some costs for external 
professional learning to support job-embedded professional learning.” 7 
Following this recommendation would require scheduling more than 1 hour a 
day for professional learning. Pligh-performing companies also understand 
the need for substantial investments in professional learning. Randy Nelson 
of Pixar reports, “Every employee is encouraged to devote up to four hours a 
week, every week, to his or her education.” 8 


School-based PLCs were seen 
by Cobb and Jackson in a 201 1 
report 9 as a key element in an 
overall instructional system leading 
to continuous improvement of 
student learning. According to these 
researchers, a well-functioning 
PLC can play a crucial role in 
school-based professional learning, 
providing opportunities for teachers 
to collaborate in addressing 
problems, to integrate ideas and 
tools introduced in district-based 
professional development, and to 
rehearse practices. They noted that 
PLCs work best when they have 
good leaders, who set agendas, 
facilitate activities, and practice 
professional routines for interaction. 
Further, for PLCs to have an impact 
on professional growth requires 
that teachers deprivatize their 
practice — that is, that they willingly 
discuss problems they encounter in 
practice, especially in their efforts 
to implement new instructional 
approaches. 

Just having a PLC, however, does 
not guarantee improved teacher 
practice and student learning. 10 
The possible lack of effectiveness 
may be especially true for schools 
that need change the most, as staff 
may have difficulty recognizing their 
need for knowledge beyond what 
they have available among their own 
faculty members. As noted in one 
recent study, “Local knowledge is 
immediate and concrete but almost 
always incomplete and sometimes 
blind and insular.” 11 
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Further, PLCs face other potential pitfalls. A recent blogpost by Learning Forward researcher Joellen Killion summarized 
findings from a study by the Rennie Center for Education Research & Policy: 

Professional learning communities (PLCs) are hijacked in multiple ways, usually under the pretense of 
facilitating or supporting . . . collaboration. Administrators who dictate the content of collaboration are 
some of the biggest offenders. Teachers who fail to engage responsibly as professionals with colleagues in 
collaboration are also offenders. When educators at any level arrive late, break commitments, seek to maintain 
the status quo, or remain within their comfort zone, they are subverting the core principles of professional 
learning communities , 12 

There also seems to be confusion about what PLCs actually are. Killion further explains that the term is often used for any 
meeting of education professionals, “Grade-level team or department meetings, faculty meetings, convocations, training, 
data presentations, curriculum writing, assessment scoring, or lesson planning are often mistakenly called PLCs.” 13 

An analysis of the items included on West Virginia catalyst school PLC agendas revealed that in most cases professional 
learning was taking a back seat to planning for learning, especially planning for student learning. Even the number 
of administrative items on agendas had a slight edge on professional learning items. The number of items is not, 
necessarily, a measure of the amount of time being spent, but does point to the possible need to take a closer look at 
how much of the time available for PLCs is being spent on professional learning activities. 

Focus on All Seven Standards for Professional Learning 

Seven standards for professional learning frame the Learning School approach. These standards were adopted by the 
WVBE in 2012 and form the basis of Board Policy 5500, which paraphrases them as follows: 

[Professional learning that increases educator effectiveness and results for all students — 

• Occurs within learning communities committed to continuous improvement, collective responsibility, and 
goal alignment. 

• Requires skillful leadership to develop capacity, advocate, and create support systems for professional 
learning. 

• Requires prioritizing, monitoring, and coordinating resources for educator learning. 

• Uses a variety of sources and types of student, educator, and system data to plan, assess, and evaluate 
professional learning. 

• Integrates theories, research, and models of human learning into learning designs to achieve its intended 
outcomes. 

• Applies research on change and sustains support for implementation of professional learning for long-term 
change. 

• Aligns its outcomes with educator performance and student curriculum standards. 

The best measure of how well aligned schools are with the Standards for Professional Learning is the SAI2. Faculty at 
10 catalyst schools took the survey in the spring of 2016 and scored high, with mean scores across all seven standards 
ranging from 3.9 to 4.4 (by RESA region) on a 5-point scale. It is notable that six of the 10 schools that took the survey 
were located in the two highest performing RESAs in the composite analysis shown in Figure 7. The highest scores for 
the seven standards among the 10 schools were for leadership (4.4) and outcomes (4.3); lowest were learning designs 
(3.8), resources (3.9), and data (3.9). 

Although a much less precise measure, the focus groups with principals conducted at all eight RESAs did have the 
advantage of including principals and/or assistant principals from nearly all catalyst schools. An analysis of the recorded 
interviews affirmed what the SAI2 showed about schools’ focus on leadership and outcomes, and a comparative lack 
of focus on data and resources. Implementation, however, was the least-mentioned of the seven standards among this 
group of principals. The low rate of mentions of data was due to the nearly singular focus on formal and informal student 
assessment data, with little or no mention of data such as educator evaluation, early warning, and other available data 
sources. 
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Notes 


One last note about the standards: While a small minority of principals 
talked in the focus groups about the relationship between the Standards for 
Professional Learning and the High Quality Schools Standards — the latter 
being the standards used by OEPA in their performance audits — most seemed 
to view the two sets of standards as separate in the sense that focusing on 
the Standards for Professional Learning would not, necessarily, be helpful 
in meeting the standards used by OEPA. It will be important for catalyst and 
other schools to have clarity about how the two sets of standards relate to one 
other without conflating them by suggesting that if you do an effective job of 
addressing the one set — that is, the High Quality Schools Standards — you will 
have adequately addressed both sets. 

Stages of Implementation 

Perhaps the most notable finding from the WVDE and RESA stage of 
implementation ratings was the tremendous range seen across the RESAs. The 
two leading RESAs, whose schools are approaching full implementation are, 
according to the WVDE and RESA observers, a full stage or more ahead of 
schools in most of the other regions. The composite ratings showed a little less 
spread in the range of schools across RESAs, but also indicated a full point 
difference on a 5-point scale between the highest and lowest sets of schools 
by RESA. 


Recommendations 

Based on the findings from this study about implementation of the Learning 
School approach in the West Virginia catalyst schools, the following four 
recommendations seem warranted: 

1 . Provide technical assistance and other resources to aid schools 

in creating schedules with ample time built into their schedules for 
professional learning— at least 45 or more minutes four times a week. 

2. Provide tools and guidance on how to analyze the activities of PLCs, 
and make sure that these meetings of collaborative learning teams 
during the school day and week include professional learning as 
their primary focus. The professional learning should draw upon the 
expertise and experience of faculty in the individual schools and 
districts, and also upon outside expertise that can be accessed 
through a variety of professional learning experiences, including 
face-to-face training, online courses, webinars, action research, book 
studies, and other learning designs. 

3. Encourage all schools to take the SAI2, help them interpret the results 
for all seven of the Standards for Professional Learning, and translate 
their results into action plans that build on their areas of strength, while 
addressing the standards in greatest need of improvement. 

4. Provide schools with a crosswalk of the Standards for Professional 
Learning and the High Quality Schools Standards, and through 
guidance and discussion help schools discern both the relationships 
and differences among these two sets of standards. 
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